Lecture # 16 : Parallel Sorting - I

Another application of sorting for parallel computation is load–balancing. For instance, consider a collection of particles distributed in arbitrary ways in a three–dimensional space. With the work per particle approximately the same, one approach would be to simply divide the particles evenly among the processing nodes. Though this approach yields load–balance, it may result in excessive communication since there is no guarantee for locality of reference. Depending upon how the particles are ordered, particles assigned to the same processing node may be far apart, while particles close in physical space may have been assigned to nodes that are far apart. A simple solution is to sort the particles by their coordinates. In one dimension, the particle coordinates can be used directly to form the keys for sorting. But, in two or more dimensions there are many ways in which the keys can be constructed from the coordinates. First, with coordinates represented as floating–point numbers they can be converted into integers for each coordinate. Then, for say a three–dimensional problem, we do not want to order particles by forming the key simply concatenating the integers for a particles x, y and z coordinates. In this ordering, particles close with respect to the coordinate chosen as the least significant part of the key will be close also in the sorted order, while particles close with respect to the coordinate chosen as the most significant may be far apart in the sorted order. Instead, what is often made in practice is forming keys by interleaving the bits from the integers representing the coordinates of a particle. Such an ordering is known as Morton ordering [6]. Once the particles are sorted in a suitable ordering, load–balancing can be accomplished, for example, through the use of orthogonal coordinate bisection. Another popular load–balancing technique relies on recursive spectral bisection, which employs sorting of components of eigenvectors for partitioning of grids.